Goto

Collaborating Authors

 adaptive normal distribution


A Locally Adaptive Normal Distribution

Georgios Arvanitidis, Lars K. Hansen, Søren Hauberg

Neural Information Processing Systems

The underlyingmetricis,however,non-parametric.Wedevelopamaximumlikelihood algorithm to infer the distribution parameters that relies on a combination of gradient descent and Monte Carlo integration. We further extend the LAND to mixture models, andprovidethecorresponding EMalgorithm.


A Locally Adaptive Normal Distribution

Neural Information Processing Systems

The multivariate normal density is a monotonic function of the distance to the mean, and its ellipsoidal shape is due to the underlying Euclidean metric. We suggest to replace this metric with a locally adaptive, smoothly changing (Riemannian) metric that favors regions of high local density. The resulting locally adaptive normal distribution (LAND) is a generalization of the normal distribution to the manifold setting, where data is assumed to lie near a potentially low-dimensional manifold embedded in R^D. The LAND is parametric, depending only on a mean and a covariance, and is the maximum entropy distribution under the given metric. The underlying metric is, however, non-parametric. We develop a maximum likelihood algorithm to infer the distribution parameters that relies on a combination of gradient descent and Monte Carlo integration. We further extend the LAND to mixture models, and provide the corresponding EM algorithm. We demonstrate the efficiency of the LAND to fit non-trivial probability distributions over both synthetic data, and EEG measurements of human sleep.



Reviews: A Locally Adaptive Normal Distribution

Neural Information Processing Systems

The paper brings new insights into the connection between metric learning and Riemannian statistics. Having parametric and generative models that can go beyond the training data is very useful in many scenarios. However, what I am mainly concerned is its applicability to real world data which is both large scale and high-dimensional. LAND requires the computation of a (here, diagonal) covariance matrix for each data point and of its inverse, which in high dimensions are known to be hard problems. The authors do mention high-dimensionality as a possible issue (lines 265-266), but the paper would need an analysis of the complexity and limitations of the metric introduced. How high is here high-dimensional?


A Locally Adaptive Normal Distribution

Arvanitidis, Georgios, Hansen, Lars K., Hauberg, Søren

Neural Information Processing Systems

The multivariate normal density is a monotonic function of the distance to the mean, and its ellipsoidal shape is due to the underlying Euclidean metric. We suggest to replace this metric with a locally adaptive, smoothly changing (Riemannian) metric that favors regions of high local density. The resulting locally adaptive normal distribution (LAND) is a generalization of the normal distribution to the "manifold" setting, where data is assumed to lie near a potentially low-dimensional manifold embedded in R D. The LAND is parametric, depending only on a mean and a covariance, and is the maximum entropy distribution under the given metric. The underlying metric is, however, non-parametric. We develop a maximum likelihood algorithm to infer the distribution parameters that relies on a combination of gradient descent and Monte Carlo integration.


A Locally Adaptive Normal Distribution

Arvanitidis, Georgios, Hansen, Lars K., Hauberg, Søren

Neural Information Processing Systems

The multivariate normal density is a monotonic function of the distance to the mean, and its ellipsoidal shape is due to the underlying Euclidean metric. We suggest to replace this metric with a locally adaptive, smoothly changing (Riemannian) metric that favors regions of high local density. The resulting locally adaptive normal distribution (LAND) is a generalization of the normal distribution to the "manifold" setting, where data is assumed to lie near a potentially low-dimensional manifold embedded in R^D. The LAND is parametric, depending only on a mean and a covariance, and is the maximum entropy distribution under the given metric. The underlying metric is, however, non-parametric. We develop a maximum likelihood algorithm to infer the distribution parameters that relies on a combination of gradient descent and Monte Carlo integration. We further extend the LAND to mixture models, and provide the corresponding EM algorithm. We demonstrate the efficiency of the LAND to fit non-trivial probability distributions over both synthetic data, and EEG measurements of human sleep.


A Locally Adaptive Normal Distribution

Arvanitidis, Georgios, Hansen, Lars Kai, Hauberg, Søren

arXiv.org Machine Learning

The multivariate normal density is a monotonic function of the distance to the mean, and its ellipsoidal shape is due to the underlying Euclidean metric. We suggest to replace this metric with a locally adaptive, smoothly changing (Riemannian) metric that favors regions of high local density. The resulting locally adaptive normal distribution (LAND) is a generalization of the normal distribution to the "manifold" setting, where data is assumed to lie near a potentially low-dimensional manifold embedded in $\mathbb{R}^D$. The LAND is parametric, depending only on a mean and a covariance, and is the maximum entropy distribution under the given metric. The underlying metric is, however, non-parametric. We develop a maximum likelihood algorithm to infer the distribution parameters that relies on a combination of gradient descent and Monte Carlo integration. We further extend the LAND to mixture models, and provide the corresponding EM algorithm. We demonstrate the efficiency of the LAND to fit non-trivial probability distributions over both synthetic data, and EEG measurements of human sleep.